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Introduction 

The major focus of the present proposal was to examine 
psychophysiological variables related to hazardous states of 
awareness induced by monitoring automated systems. With the 
increased use of automation in today's work environment, people's 
roles in the work place are being redefined from that of active 
participant to one of passive monitor. Although the introduction 
of automated systems has a number of benefits, there are also a 
number of disadvantages regarding worker performance (Wiener, 
1988). Byrne and Parasuraman (1996) have argued for the use of 
psychophysiological measures in the development and the 
implementation of adaptive automation. While both performance 
based and model based adaptive automation have been studied (e.g. 
Parasuraman, Mouloua, & Molloy, 1996) , the use of 
psychophysiological measures, especially EEG, offers the 
advantage of real time evaluation of the state of the subject. 

The current study used the closed-loop system, developed at 
NASA-Langley Research Center, to control the state of awareness 
of subjects while they performed a cognitive vigilance task. 
Previous research in our laboratory, supported by NASA, has 
demonstrated that, in an adaptive automation, closed-loop 
environment, subjects perform a tracking task better under a 
negative than a positive, feedback condition. In addition, this 



condition produces less subjective workload and larger P300 event 
related potentials to auditory stimuli presented in a concurrent 
oddball task. We have also recently shown that the closed-loop 
system used to control the level of automation in a tracking task 
can also be used to control the event rate of stimuli in a 
vigilance monitoring task. By changing the event rate based on 
the subject's index of arousal, we have been able to produce 
improved monitoring, relative to various control groups. 

We have demonstrated in our initial closed-loop experiments 
with the the vigilance paradigm that using a negative feedback 
contingency (i.e. increasing event rates when the EEG index is 
low and decreasing event rates when the EEG index is high) 
results in a marked decrease of the vigilance decrement over a 
40 minute session. This effect is in direct contrast to 
performance of a positive feedback group, as well as a number of 
other control groups which demonstrated the typical vigilance 
decrement. Interestingly, however, the negative feedback group 
performed at virtually the same level as a yoked control group. 
The yoked control group received the same order of changes in 
event rate that were generated by the negative feedback subjects 
using the closed-loop system. Thus it would appear to be 
possible to optimize vigilance performance by controlling the 
stimuli which subjects are asked to process. 



EXPERIMENT 1: 


The experiment conducted had subjects perform a vigilance 
monitoring task. The task was cognitive in nature and adopted the 
visual search paradigm employed by Treisman (1988). In this 
paradigm subjects were presented with displays containing 
multidimensional objects, consisting of colored letters. Stimulus 
trials were always presented at the rate of once every five 
seconds (i.e. 12 trials per minute) with either two, five or nine 

colored letters being shown on a given trial. The target 
stimulus was always a green "K" which was presented on one trial 
per minute at random times within that minute. The stimuli 
consisted of either an "N", "K", "R", or "X" presented at random 
positions on the computer screen with the only stipulation being 
that the stimuli did not overlap each other. The color of the 
stimuli was either green, yellow-green or blue-green, except for 
the "K" which was either yellow-green or blue-green on non-target 
trials. Under the negative feedback condition, when a subject's 
EEG index was at least 0.2 standard deviations above baseline 
they were presented with a simple feature search condition in 
which only two stimuli were presented on the computer screen. The 
subject was instructed to respond, by pressing a button only when 
the target stimulus appeared. If the subject's index was within 
+/- 0.2 standard deviations of the baseline five different 
stimuli were presented on the computer screen. If the subject's 
index was more than 0.2 standard deviations below their baseline, 



9 stimuli were displayed. Research by Treisman and her associates 
(Treisman, 1988; Treisman & Gelade, 1980) has demonstrated that 
conditions which require a "conjunctive search" are more 
difficult than simple feature searches because they require 
processing that involves multidimensional percepts. The more 
stimuli presented, the more processing that is required and the 
longer the reaction times. Thus, under negative feedback, 
requiring subjects to do more cognitive processing when their 
index is low should produce greater task involvement and a 
subsequent increase in the engagement index. Under positive 
feedback, the reverse contingencies will be in place and should 
result in poorer task performance and a reverse relationship 
between stimulus conditions and the engagement index. 

EEG was recorded continuously from four sites (F3, F4, 01 

and 02, using the international 10-20 system) during the 40 
minute task and fed into a Lab— View virtual instrument which 
calculated the p/ (a+0) index. These sites were chosen because 1. 
the frontal lobes have been theorized to be involved in 
attent ional processing (cf. Lubar, 1991; Posner & Raichle, 1994) 
and 2. the occipital lobes generating high levels of theta during 
lapses in attention (e.g. Beatty & O'Hanlon, 1979). 

Subjects were brought into the laboratory and explained the 
nature of the experiment. They were then asked sign an informed 
consent, indicating that they understood the purpose and 
procedure of the study. The electrode cap was then placed on the 


subject's head and a reference electrode placed on the left 
mastoid. All electrode impedances were lowered to below 5kohms. 
The subjects were then placed in front of the task computer and 
given six trials that demonstrated the nature of the stimuli that 
would be used in the task. If requested, the subject could 
receive six more trials to insure they understood what they would 
be looking for. They were then given an 8 -minute practice trial 
in which the three different stimulus arrays were randomly 
p£gg 0 rip 0 (j f once every five seconds, with a target trial occurring 
once per minute. At the end of the eight minute practice the 
subject's A' score was determined. If A' was at least . 8 , the 
regular 40-minute session was begun. If the A' score was below 
. 8 , the subject was given a second eight minute practice session. 
If their score was still below . 8 , they were not used in the 
experiment. All subjects that completed the experiment were 
either given two hours of extra credit for their class or $ 20 . 00 , 
whichever they preferred. Subjects that did not get past the 
practice were given either one hour extra credit or $ 5 . 00 . 

Results 

First, A' scores were calculated to assess performance. In 
Table 1 is presented the ANOVA for the A' scores. The only 
significant effect was for periods, F ( 3 , 8 7 ) = 16.6, p < . 001 . 

The scores are plotted in Figure 1 . It can be seen that A 
improved for both positive and negative feedback groups from the 


first 10-minute block to the second 10-minute block. While the 
scores for the negative feedback group continued to improve 
slightly over the remainder of the session, those of the positive 
feedback group declined slightly. In Table 2 is presented the 
ANOVA for B" scores. Again, the only significant effect was for 
periods , F (3,87) = 17.1, p < .01. As seen in Figure subjects 
became more conservative as the session progressed, regardless of 
which group they were in. 

To account for the lack of group differences observed in the 
A' and B" scores, we examined the false alarm and probability of 
hit data. In Table 3 is presented the ANOVA for false alarms. The 
only significant effect was for periods, F (3, 87) = 75.3, p < 
.01. As can be seen in Figure 3, the false alarm rate was 
relatively high for the first 10-minute period, but then dropped 
to close to zero for the last three 10-minute periods. Thus, the 
A' and B" data can be accounted for by the fact that the subjects 
appeared to still be learning the task for the first 10-minute 
period. Since false alarms dropped to near zero after that time, 
it can be argued that the only true assessment of performance is 
the probability of a hit. A’ scores are not appropriate if 
subjects' false alarm rate is virtually zero. 

In Table 4 is presented the ANOVA for probability of a hit 
for the negative and positive feedback groups. A significant 
group effect was found F (1,29) = 4.21, p < .05. In Figure 4 are 
presented the data for probability of a hit. It can be seen that 


the negative feedback group improved slightly from the first to 
the last 10-minute period while the positive feedback group's 
performance decreased dramatically over the session. Curiously, 
the interaction between groups and periods was not significant, 
F(3, 87) = 1.88, p > .10. Apparently, the variability across 

groups was large enoug.o to washout any effect. 

The analysis of the EEG index data is presented in Table 5. 
The critical effect that needs to be examined is the group by 
array size interaction. In all of our previous studies, 
including tracking and perceptual vigilance, there was a 
significant interaction between feedback and either the 
manual/automatic mode (for tracking) or the stimulus rate (for 
perceptual vigilance) . This interaction is a basic validation of 
the system's functioning. When subjects in the negative feedback 
condition become underaroused, the system should switch them to 
the condition that increases arousal. Conversely, when they are 
overa roused , the system should switch them to the opposite 
condition. For the positive feedback group, if subjects are 
highly aroused the system should keep them in that condition, and 
vice versa. The group by array size interaction was found to be 
significant, F(2,40) = 97.7, p < .001. As seen in Figure 5, the 
typical crossover effect that we have found in all of our earlier 
experiments was also produced by the cognitive vigilance task. 

In Figure 6 is presented this interaction across periods (F(6, 
120) =7.92, p < .01. 



Finally, we compared the amount of time each group spent at 
each array size. We had originally reasoned that the smal_est 
array size (i.e. two stimuli) would be the easiest to process and 
thus would not cause any decrement. Conversely, we expected that 
when subjects tried to process the large array (i.e. nine 
stimuli) their performance would deteriorate over the 40 minute 
session due to greater fatigue induced by having to constantly 
search for the target. Since the positive feedback group did 
significantly poorer than the negative feedback group, we assumed 
the positive feedback group probably spent more time (as 
reflected by number of non-target presentations) with the larger 
array size. The results of the ANOVA for number of non-target 
presentations is presented in Table 6. As expected, there was a 
significant group effect (F (1,20) = 4.66 p < .05) and a 
significant array size effect (F (2, 40) = 26.4, p < .001, 1 . 

There was also a significant period effect (F (3, 60) = 8.2, p < 

.001). There was only one significant interaction, the group by 
array size (F (2, 40) = 22.7, p < .001) . This interaction is 
presented in Figure 7 . As can be seen in the figure, the results 
were the exact opposite of what we expected. The negative 
feedback group, which had performed significantly better, as 
measured by the probability of a hit, spent the majority of the 
time with the large array, while the positive feedback group 
spent the majority of the time with the small array. Neither 
group spent much time at the middle array size. 


Discussion . 


The results of the first experiment have demonstrated that 
negative feedback in the closed-loop psychophysiological system 
can improve performance on a cognitive vigilance task just as it 
can for a motor task (manual tracking) and perceptual vigilance 
task (line length) . Further, changes in the EEG index, which 
validate system functioning, are comparable across all three 
tasks. Surprisingly, the stimulus condition which produced the 
g^03p05t vigilance decrement was the two— stimulus array. It 
would seem that when reguiring subjects to continuously engage in 
a visual search task, presenting subjects with more stimuli to 
search, rather than inducing fatigue, kept subjects sufficiently 
aroused to maintain a high level of performance. Apparently, a 
two-stimulus array, while easy to search for a target, is not 
sufficiently stimulating, resulting in a vigilance performance 
decrement . Since this type of cognitive vigilance paradigm has 
never been used before, we felt it was necessary to confirm that, 
in a non-closed loop design, a vigilance decrement would indeed 
be seen with a two, but not a nine stimulus array, cognitive 
vigilance task. Experiement two tested this hypothesis. 

Experiment 2 

In Experiment 2 , two groups were run: group one was always 
presented the two-stimulus array while group two was always 


presented the nine stimulus array. All other stimulus parameters, 
intervals , electrode sites, EEG index, etc. , were the same as in 
Experiment 1. 

Results 

In Table 8 is presented the ANOVA for the A' scores. There 
was a significant effect for both periods (F (3, 87) = 20.1, £ < 
.001) and for the group by period interaction (F ( 3, 87) = 3.79, 
p < .02) . As was seen for Experiment 1, A' scores, presented in 
Figure 9, were lowest in the first 10-minute period for both 
groups. However, the nine— stimulus array group maintained its 
high performance level for the remainder of the session while the 
two stimulus array group's performance began to decrease. 

The poorer performance for the first 10-minute period can 
again be attributed to the high false alarm rate for that time. 

In Table 9 is presented the ANOVA for the false alarms. The only 
significant effect is for periods (F (3, 87) = 40.2, p < .001). 

As seen in Figure 10, false alarms were high for the first 10 
minutes then decreased to approximately zero for the remainder of 
the task. Because of the low false alarm rate for the majority 
of the task, we again felt that the probability of a hit best 
reflected subject performance. 

In Table 10 is presented the ANOVA for probability of a hit 
for the two groups. The only significant effect was the 
interaction between groups and period (F (3, 8/) = 3.67, p < 

.02). As seen in Figure 11, for the nine-stimulus array group, 



performance was slightly lower than that of the two-stimulus 
array group for the first period. However, the nine-stimulus 
group improved its performance slightly over the experimental 
session while the two-stimulus group showed a vigilance 
decrement . 

Finally, the EEG index was analyzed. Despite the strong 
differences observed in Experiment 1, there were no significant 
differences involving either group or period for Experiment 2. 
All Fs were below 1.0. 

Discussion 

The results of Experiment 2 confirmed the hypothesis that 
the vigilance decrement seen in the positive feedback group of 
Experiment 1 may be attributed to the fact that this group spent 
approximately two-thirds of the experimental session observing 
the two-stimulus array. That is, the positive feedback subjects 
became under aroused by the task, and because of their task 
contingencies (i.e. continue to show the two— stimulus array if 
the EEG index is low) caused them to remain in a lowered 
arousal/EEG index condition. Because the negative feedback 

were reguired to engage in more cognitive functioning 
(i.e. constantly search the more complex array for the target) 
their arousal level/EEG index remained higher and their 
performance did not deteriorate. 

It should be noted that the reason the subjects typically 
had such a high false alarm rate for the first 10 minutes of the 



task may be due to setting the cutoff for being in the experiment 
at a ' —> .8. If Figures 1 and 9 are examined, it will be noted 
that even thought the false alarm rates were high for the first 
10 minutes, A' scores were still no lower than .9. Thus, 
subjects, on average, were performing well above the cutoff, even 
for the first period. 

Finally, the value of the EEG index needs to be addressed. 
Although the closed-loop adaptive automation system drove the EEG 
index as expected, without any feedback, the index remained 
relatively stable. In other words, it did not appear to be the 
nature of the stimuli or the amount of cognitive processing that 
is important, but rather, something about the contingencies of 
the closed-loop system. Since the task, regardless of the size 
of the stimulus array, would seem to be boring in nature, one 
might expect low indexes. Only with the closed-loop were we able 
to drive the index up or down. We have never examined how the 
index would be affected by a truly arousing task. We have done 
some pilot work using the MAT battery where subjects were 
required to perform three tasks at one time. Although subjects 
reported higher workloads compared to just performing one task, 
the EEG indexes still did not reflect any differences in 
conditions. However, even in that pilot study, it is not clear 
whether the high workload was due to the boring nature of the 
task. It would be of interest to look at how the index might be 
affected by a truly highly stressful task. 
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Table 1. A' Scores Closed-loop Experiment 1. 


SS df MS F 

Group 1 .017 2.82 

Period 3 .028 16.6 

GroupxPer 3 .001 .85 


£ 

. 10 
.0001 
. 47 



Table 2. B" Scores Closed-loop Experiment 1. 


SS df MS F 

Group 1 .183 .227 

Period 3 5.44 17.1 

GroupxPer 3 .086 .27 


£ 

. 637 

.0001 

.84 



Table 3. False Alarms Closed-loop Experiment 1. 


SS df MS F 

Group 1 .0057 .055 
Period 3 .356 75.3 
GroupxPer 3 .00003 .007 


£ 

.81 
.0001 
. 99 



Table 4. Probability of a Hit Closed-loop Experiment 1. 


SS df MS F 2 

Group 1 >19 4.2 .049 

Period 3 .014 .97 .41 

GroupxPer 3 .028 1.88 .14 



Table 5. EEG Index Scores Closed-loop Experiment 1. 


SS df MS E. 


Group 1 

1.07 

.01 

Period 3 

31.87 

2.67 

Array 2 

52.4 

2.67 

Grp x Ar 2 

1914 . 6 

97 . 7 

GrpxPxAr 6 

57.4 

7 . 9 


£ 

.91 
. 054 
. 08 
. 0001 
.0001 


Table 6. Time spent at Each Array Closed-loop Experiment 1. 


SS 

Group 

Period 

Array 

GrpxAr 

GrpxPxAr 


df 

1 

3 

2 

2 

6 


MS 

. 93 

3.3 

27413.3 

23499.3 
134 . 8 


F B 


4 . 66 

.04 

8.2 

.0001 

26.4 

.0001 

22 . 7 

.0001 

1.3 

.25 



Table 7. A' Scores 9 vs 2 Array Experiment 2. 


SS df MS F E 

Group 1 .0004 .05 .8 

Period 3 .017 20.1 .0 

GroupxPer 3 .003 3.8 .013 


o r ) 



Table 8. False Alarms 9 vs 2 Array Experiment 2. 


SS df MS F 

Group 1 .0005 .05 

Period 3 .176 40.2 

Group x per 3 .003 .62 


£ 

.82 
.0001 
. 60 



Table 9. Probability of a Hit 9 vs 2 Array Experiment 2. 


SS df MS 

Group 1 • 02 

Period 3 .01 

Group x per 3 .023 


.20 
2 . 02 
3.67 


. 65 
.11 
.015 


Table 10. EEG Index Scores 9 vs 2 Array Experiment 2. 


SS M MS 

Group 1 2.1 

Period 3 .49 

Array 2 3.19 

Grp x Ar 2 .01 

GrpxPxAr 6 1.56 


F E 

.01 .91 

.14 .92 

2.2 .12 

.008 .99 

.69 .66 


